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ABSTRACT 

Stability is perhaps one of the most desirable features of 
any engineered system, given the importance of being able 
to predict its response to various environmental conditions 
prior to actual deployment. Engineered systems are be- 
coming ever more complex, approaching the same levels of 
biological ecosystems, and so their stability becomes ever 
more important, but taking on more and more differential 
dynamics can make stability an ever more elusive property. 
The Chli-DeWilde definition of stability views a Multi- 
Agent System as a discrete time Markov chain with poten- 
tially unknown transition probabilities. With a Mult i- Agent 
System being considered stable when its state, a stochastic 
process, has converged to an equilibrium distribution, be- 
cause stability of a system can be understood intuitively as 
exhibiting bounded behaviour. We investigate an extension 
to include Multi-Agent Systems (MASs) with evolutionary 
dynamics, focusing on the evolving agent populations of our 
Digital Ecosystem. We then built upon this to construct 
an entropy-based definition for the degree of instability 
(entropy of the limit probabilities), which was later used 
to perform a stability analysis. The Digital Ecosystem is 
considered to investigate the stability of an evolving agent 
population through simulations, for which the results were 
consistent with the original Chli-DeWilde definition. 
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1. INTRODUCTION 

Digital Ecosystems are distributed adaptive open socio- 
technical systems, with properties of self-organisation, scal- 
ability and sustainability, inspired by natural ecosystems 
[3], and are emerging as a novel approach to the catalysis 
of sustainable regional development driven by Small and 
Medium sized Enterprises (SMEs). Digital Ecosystems aim 
to help local economic actors become active players in 
globalisation, valorising their local culture and vocations, 
and enabling them to interact and create value networks at 
the global level [12]. With its technical component being the 
digital counterpart of a biological ecosystem, providing for 
the evolution of software services (agents) in a distributed 
network [7, 6]. 

Multi- Agent Systems (MASs) is a growing field, primarily 
because of recent developments of the Internet as a means 
of circulating information, goods and services; and many 
researchers have contributed valuable work in the area 
in recent years [26]. However, despite both evolutionary 
computing and MASs being mature research areas [24, 18] 
their integration is a recent development [31] and non- 
trivial, because agents can be modelled as state machines 
and evolutionary computing algorithms have been developed 
to work on numerical data and strings without memory 
effects. Their integration also allows for the creation of 
the digital counterpart of a biological ecosystem, what we 
call a Digital Ecosystem [5], which provides a conceptual 
architecture for the evolution of software agents (services) in 
a distributed network [4]. Our aim here is to determine, for 
our Digital Ecosystem and other MASs which make use of 
evolutionary computing [17, 30], macroscopic variables that 
characterise the stability of their evolving agent populations 
and therefore the system as a whole. 

2. THE DIGITAL ECOSYSTEM 

Our Digital Ecosystem [5] provides a two-level optimi- 
sation scheme inspired by natural ecosystems, in which a 
decentralised peer-to-peer network forms an underlying tier 
of distributed agents. These agents then feed a second 
optimisation level based on an evolutionary algorithm that 
operates locally on single habitats (peers), aiming to find 
solutions that satisfy locally relevant constraints. The local 
search is sped up through this twofold process, providing 
better local optima as the distributed optimisation provides 
prior sampling of the search space by making use of 
computations already performed in other peers with similar 
constraints [4]. So, the Digital Ecosystem supports the au- 





Figure 1: Digital Ecosystem: Optimisation 
architecture in which agents (representing services) 
travel along the P2P connections; in every node 
(habitat) local optimisation is performed through 
an evolutionary algorithm, where the search space 
is determined by the agents present at the node. 



tomatic combining of numerous agents (which represent ser- 
vices), by their interaction in evolving populations to meet 
user requests for applications, in a scalable architecture of 
distributed interconnected habitats. The sharing of agents 
between habitats ensures the system is scalable, while 
maintaining a high evolutionary specialisation for each user. 
The network of interconnected habitats is equivalent to 
the abiotic environment of biological ecosystems; combined 
with the agents, the populations, the agent migration for 
distributed evolutionary computing, and the environmental 
selection pressures provided by the user base, then the 
union of the habitats creates the Digital Ecosystem, which 
is summarised in Figure 1. The continuous and varying user 
requests for applications provide a dynamic evolutionary 
pressure on the applications (agent aggregations), which 
have to evolve to better fulfil those user requests, and 
without which there would be no driving force to the 
evolutionary self-organisation of the Digital Ecosystem. 

If we consider an example user base for the Digital Ecosys- 
tem, the use of Service- Oriented Architectures (SOAs) 
in its definition means that business-to-business (B2B) 
interaction scenarios lend themselves to being a potential 
user base for Digital Ecosystems. So, we can consider a 
business ecosystem of Small and Medium sized Enterprise 
(SME) networks [20], as a specific class of examples for 
B2B interaction scenarios; and in which the SME users are 
requesting and providing software services, represented as 
agents in the Digital Ecosystem, to fulfil the needs of their 
business processes, creating a Digital Business Ecosystem 
as shown in Figure 2. SOAs promise to provide potentially 
huge numbers of services that programmers can combine, 
via the standardised interfaces, to create increasingly more 
sophisticated and distributed applications. The Digital 
Ecosystem extends this concept with the automatic com- 
bining of available and applicable services, represented by 
agents, in a scalable architecture, to meet user requests 
for applications. These agents will recombine and evolve 
over time, constantly seeking to improve their effectiveness 
for the user base. From the SME users' point of view the 
Digital Ecosystem provides a network infrastructure where 
connected enterprises can advertise and search for services 
(real- world or software only) , putting a particular emphasis 
on the composability of loosely coupled services and their 
optimisation to local and regional, needs and conditions. To 
support these SME users the Digital Ecosystem is satisfying 
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Figure 2: Digital Business Ecosystem: Business 
ecosystem, network of SMEs [20], using the Digital 
Ecosystem. The habitat clustering will therefore be 
parallel to the business sector communities. 



the companies' business requirements by finding the most 
suitable services or combination of services (applications) 
available in the network. An application (composition of 
services) is defined be an agent aggregation (collection) in 
the habitat network that can move from one peer (company) 
to another, being hosted only in those where it is most useful 
in satisfying the SME users' business needs. 

The agents consist of an executable component and an 
ontological description. So, the Digital Ecosystem can 
be considered a MAS which uses distributed evolutionary 
computing to combine suitable agents in order to meet user 
requests for applications. 

The landscape, in energy-centric biological ecosystems, 
defines the connectivity between habitats. Connectivity 
of nodes in the digital world is generally not defined 
by geography or spatial proximity, but by information 
or semantic proximity. For example, connectivity in a 
peer-to-peer network is based primarily on bandwidth and 
information content, and not geography. The island- models 
of distributed evolutionary computing use an information- 
centric model for the connectivity of nodes (islands) [16]. 
However, because it is generally defined for one-time use (to 
evolve a solution to one problem and then stop) it usually 
has a fixed connectivity between the nodes, and therefore 
a fixed topology. So, supporting evolution in the Digital 
Ecosystem, with a multi-objective selection pressure (fitness 
landscape with many peaks), requires a re- configurable 
network topology, such that habitat connectivity can be 
dynamically adapted based on the observed migration paths 
of the agents between the users within the habitat network. 
Based on the island-models of distributed evolutionary 
computing [16], each connection between the habitats is 
bi-directional and there is a probability associated with 
moving in either direction across the connection, with 
the connection probabilities affecting the rate of migration 
of the agents. However, additionally, the connection 
probabilities will be updated by the success or failure of 
agent migration using the concept of Hebbian learning: 
the habitats which do not successfully exchange agents 
will become less strongly connected, and the habitats 
which do successfully exchange agents will achieve stronger 
connections. This leads to a topology that adapts over 
time, resulting in a network that supports and resembles 
the connectivity of the user base. If we consider a business 
ecosystem, network of SMEs, as an example user base; 
such business networks are typically small- world networks. 
They have many strongly connected clusters (communities), 
called sub-networks (quasi-complete graphs), with a few 



connections between these clusters (communities). Graphs 
with this topology have a very high clustering coefficient 
and small characteristic path lengths [33]. So, the Digital 
Ecosystem will take on a topology similar to that of the user 
base, as shown in Figure 2. 

The novelty of our approach comes from the evolving 
populations being created in response to similar requests. 
So whereas in the island-models of distributed evolutionary 
computing there are multiple evolving populations in re- 
sponse to one request [16], here there are multiple evolving 
populations in response to similar requests. In our Digital 
Ecosystems different requests are evaluated on separate 
islands (populations), and so adaptation is accelerated 
by the sharing of solutions between evolving populations 
(islands) , because they are working to solve similar requests 
(problems) . 

The users will formulate queries to the Digital Ecosystem 
by creating a request as a semantic description, like those 
being used and developed in SOAs, specifying an application 
they desire and submitting it to their local peer (habitat). 
This description defines a metric for evaluating the fitness 
of a composition of agents, as a distance function between 
the semantic description of the request and the agents' 
ontological descriptions. A population is then instantiated 
in the user's habitat in response to the user's request, 
seeded from the agents available at their habitat. This 
allows the evolutionary optimisation to be accelerated in the 
following three ways: first, the habitat network provides a 
subset of the agents available globally, which is localised 
to the specific user it represents; second, making use 
of applications (agent aggregations) previously evolved in 
response to the user's earlier requests; and third, taking 
advantage of relevant applications evolved elsewhere in 
response to similar requests by other users. The population 
then proceeds to evolve the optimal application (agent 
aggregation) that fulfils the user request, and as the agents 
are the base unit for evolution, it searches the available 
agent combination space. For an evolved agent aggregation 
(application) that is executed by the user, it then migrates 
to other peers (habitats) becoming hosted where it is useful, 
to combine with other agents in other populations to assist 
in responding to other user requests for applications. 

3. AGENT STABILITY 

While there are several definitions of stability [21, 34, 
25] defined for MASs, they are not applicable primarily 
because of the evolutionary dynamics inherent in the con- 
text of evolving agent populations. Whereas Chli-DeWilde 
stability of MASs [9] may be suitable, because it models 
MASs as Markov chains, which are an established modelling 
approach in evolutionary computing [28] . A MAS is viewed 
as a discrete time Markov chain with potentially unknown 
transition probabilities, in which the agents are modelled as 
Markov processes, and is considered to be stable when its 
state, a stochastic process, has converged to an equilibrium 
distribution [9]. Also, while there has been past work on 
modelling evolutionary computing algorithms as Markov 
chains [27, 22, 14, 13], we have found none including MASs 
despite both being mature research areas [24, 18], because 
their integration is a recent development [31]. We therefore 
decided to extend the existing Chli-DeWilde definition 
of agent stability to include the necessary evolutionary 
dynamics. 



Chil-DeWilde stability was created to provide a clear 
notion of stability in MASs [9], because stability is perhaps 
one of the most desirable features of any engineered system, 
given the importance of being able to predict its response 
to various environmental conditions prior to actual deploy- 
ment; and while computer scientists often talk about stable 
or unstable systems [32, 2], they did so without having 
a concrete or uniform definition of stability. Also, other 
properties had been widely investigated, such as openness 
[1], scalability [19] and adaptability [29], but stability 
had not. So, the Chli-DeWilde definition of stability for 
MASs was created [9] , based on the stationary distribution 
of a stochastic system, modelling the agents as Markov 
processes, and therefore viewing a MAS as a discrete 
time Markov chain with a potentially unknown transition 
probability distribution. The MAS is considered to be 
stable once its state, a stochastic process, has converged 
to an equilibrium distribution [9], because stability of a 
system can be understood intuitively as exhibiting bounded 
behaviour. 

Chli-DeWilde stability was derived [8] from the notion 
of stability defined by De Wilde [11, 15], based on the 
stationary distribution of a stochastic system, making use of 
discrete-time Markov chains, which we will now introduce. 
If we let / be a countable set, in which each i G / is called a 
state and / is called the state-space. We can then say that 
A = (Xi : z G /) is a measure on / if < < oo for all 
i ^ I, and additionally a distribution if X^^^j A^ = 1 [8]. So, 
if X is a random variable taking values in / and we have 
Xi — Fy(X = i), then A is the distribution of X, and we 
can say that a matrix P = {pij : i,j G /) is stochastic if 
every row (pij : j G /) is a distribution [8]. We can then 
extend familiar notions of matrix and vector multiplication 
to cover a general index set / of potentially infinite size, by 
defining the multiplication of a matrix by a measure as AP, 
which is given by 

{XP)^ = J2>^JP^j^ (1) 

We can now describe the rules for a Markov chain by a 
definition in terms of the corresponding matrix P [8]. 

Definition 1. We say that {X^)t>o is a Markov chain 
with initial distribution A = (Ai : z G /) and transition 
matrix P = {pij : i,j ^ I) if: 

1, Pr(X° = io) = XiQ and 

2. Pr(X*+i = it+i I X° = zo, . . . , X* = it) = PH^,+, . 
We abbreviate these two conditions by saying that {X^)t>o 

is Markov(X, P). 

In this first definition the Markov process is memoryless, 
resulting in only the current state of the system being 
required to describe its subsequent behaviour. We say that a 
Markov process X^ , . . . , X* has a stationary distribution 
if the probability distribution of X* becomes independent 
of the time t [9]. So, the following theorem is an easy 
consequence of the second condition from the first definition. 

Theorem 1. A discrete-time random process {X^)t>o is 
Markov{X, P), if and only if for all t and io, . . . ,it we have 

Pr(X° =zo,...,X* =zt) = Xi^pi^i^ '"Pit-iif (2) 



This first theorem depicts the structure of a Markov 
chain, illustrating the relation with the stochastic matrix 
P, and defining its time-invariance property [8]. 

Theorem 2. Let {X*)t>o be Markov {X, P), then for all 
t,s > 0; 

1, Pr(X* = j) = {XP')j and 

2. Ft(X' = j \ X"" = i) = Pr(X*+^ = j \ X' = i) = 
For convenience {P^)ij can be more conveniently denoted as 



Given this second theorem we can define p^^j' as the t-step 
transition probability from the state i to j [8], and we can 
now introduce the concept of an invariant distribution [8], 
in which we say that A is invariant if 

XP = X. (3) 

The next theorem will link the existence of an invariant 
distribution^ which is an algebraic property of the matrix P, 
with the probabilistic concept of an equilibrium distribution. 
This only applies to a restricted class of Markov chains, 
namely those with irreducible and aperiodic stochastic 
matrices. However, there is a multitude of analogous results 
for other types of Markov chains to which we can refer [23, 
10], and the following theorem is provided as an indication 
of the family of theorems that apply. An irreducible matrix 
P is one for which, for all i,jGl there are sufficiently large 
t^Pif > is aperiodic if for all states z G / we have 

p\f > for all sufficiently large t [8]. 

Theorem 3. Let P be irreducible, aperiodic and have 
an invariant distribution, X can be any distribution, and 
suppose that {X*)t>o is Markov{X,P) [8], then 

Pr(X* = j)^p]° as t^ 00 for all j el (4) 

and 

Pif ~^ pT cls t ^ 00 for all i, j G /. (5) 

We can now view a system 5' as a countable set of states 
/ with implicitly defined transitions P between them, and 
at time t the state of the system is the random variable X*, 
with the key assumption that {X*)t,o is Markov(A, P) [8]. 

Definition 2. The system S is said to be stable when 
the distribution of the its states converge to an equilibrium 
distribution^ 

Pr(X* = j) pY as t ^ 00 for allj G /. (6) 

More intuitively, the system a stochastic process 
is stable if the probability distribution of X* 
becomes independent of the time index t for large t [9]. 
Most Markov chains with a finite state-space and positive 
transition probabilities are examples of stable systems, 
because after an initialisation period they settle down on 
a stationary distribution [8] . 

A MAS can be viewed as a system S, with the system 
state represented by a finite vector X, having dimensions 



large enough to manage the agents present in the system. 
The state vector will consist of one or more elements for each 
agent, and a number of elements to define general properties 
of the system state. We can then model an agent as being 
dead, i.e. not being present in the system, by setting the 
vector elements for that agent to some predefined null value 
[8]. 

3.1 Extensions for Evolving Populations 

Extending Chli-DeWilde stability to the class of MASs 
that make use of evolutionary computing algorithms, includ- 
ing our evolving agent populations, requires consideration 
of the following issues: the inclusion of population dynamics, 
and an understanding of population macro-states. 

3.1.1 Population Dynamics 

First, the MAS of an evolving agent population is com- 
posed of n agent aggregations, with each agent aggregation 
z in a state at time t, where i = 1, 2, . . . , n. The states 
of the agent aggregations are random variables, and so the 
state vector for the MAS is a vector of random variables 
with the time being discrete, t = 0,1, . . . . The interactions 
among the agent aggregations are noisy, and are given by 
the probability distributions 

Pr(Xi|Y) = Pt{C+' =Xi\e=Y), l,...,n, (7) 

where Xi is a value for the state of agent aggregation i, 
and Y is a value for the state vector of the MAS. The 
probabilities implement a Markov process, with the noise 
caused by mutations. Furthermore, the agent aggregations 
are individually subjected to a selection pressure from the 
environment of the system, which is applied equally to 
all the agent aggregations of the population. So, the 
probability distributions are statistically independent, and 

Pr(X|Y) = n?=i Pr(^|+i = Xi\e = Y). (8) 

If the occupation probability of state X at time t is denoted 
by px, then 

pl, = J2PT{X\Y)pt^\ (9) 

Y 

This is a discrete time equation used to calculate the 
evolution of the state occupation probabilities from t = 0, 
while equation (8) is the probability of moving from one 
state to another. The MAS (evolving agent population) 
is self-stabilising if the limit distribution of the occupation 
probabilities exists and is non-uniform, i.e. 

p^ = limt^ooPx. (10) 

exists for all states X, and there exist states X and Y such 
that 

Px#P?. (11) 

These equations define that some configurations of the 
system, after an extended time, will be more likely than 
others, because the likelihood of their occurrence no longer 
changes. Such a system is stable, because the likelihood 
of states occurring no longer changes with time, and is 
the definition of stability developed in [9]. While equation 
(10) is the probabilistic equivalence of an attractor^ in a 

^An attractor is a set of states, invariant under the dy- 
namics, towards which neighbouring states asymptotically 
approach during evolution. 



system with deterministic interactions, which we had to 
extend to a stochastic process because mutation is inherent 
in evolutionary dynamics. 

Although the number of agents in the Chli-DeWilde 
formalism can vary, we require it to vary according to the se- 
lection pressure acting upon the evolving agent population. 
We must therefore formally define and extend the definition 
of dead agents, by introducing a new state d for each agent 
aggregation. If an agent aggregation is in this state, = 
then it is dead and does not affect the state of other agent 
aggregations in the population. If an agent aggregation i 
has low fitness then that agent aggregation will likely die, 
because 

Pr(d|Y) = Pr(Cr^ = d\e = Y) (12) 

will be high for all Y. Conversely, if an agent aggregation 
has high fitness, then it will likely replicate, assuming the 
state of a similarly successful agent aggregation (mutant), 
or crossover might occur changing the state of the successful 
agent aggregation and another agent aggregation. 

3.1.2 Population Macro-States 

As we defined earlier, the state of an evolving agent popu- 
lation is determined by the collection of agent aggregations 
of which it consists at a specific time t, and potentially 
changing state as the time t increases. So, we can define a 
macro-state M as a set of states with a common property, 
here possessing at least one copy of the current maximum 
fitness individual. Therefore, by its definition, each macro- 
state M must also have a maximal state composed entirely 
of copies of the current maximum fitness individual. There 
must also be a macro-state consisting of all the states 
that have at least one copy of the global maximum fitness 
individual, which we will call the maximum macro-state 
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We can consider the macro-states of an evolving agent 
population visually through the representation of the state- 
space / of the system S shown in Figure 3, which includes 
a possible evolutionary path through the state-space /. 
Traversal through the state-space / is directed by the 
selection pressure of the evolutionary process acting upon 
the population S, driving it towards the maximal state of 
the maximum macro-state Mmax, which consists entirely 



maximum macro-state M^ 




Figure 3: State- Space of an Evolving Agent 
Population: A possible evolutionary path through 
the state-space / is shown, with the selection 
pressure of the evolutionary process driving it 
towards the maximal state of the maximum macro- 
state Mmax' 



of copies of the optimal solution, and is the equilibrium 
state that the system S is forever falling towards without 
ever quite reaching, because of the noise (mutation) within 
the system. So, while this maximal state will never be 
reached, the maximum macro-state Mmax itself is certain 
to be reached, provided the system does not get trapped at 
local optima, i.e. the probability of being in the maximum 
macro-state Mmax at infinite time is one, PM^ax = I5 as 
defined from equation (9). 

Furthermore, we can define quantitatively the probability 
distribution of the macro-states that the system occupies at 
infinite time. For a stable system, as defined by equation 
(11), the degree of instability, dins, can be defined as the 
entropy of its probability distribution at infinite time, 

dins H{p°^) - ^Px ^o^iv(px ), (13) 

X 

where N is the number of possible states, and taking log to 
the base N normalises the degree of instability. The degree 
of instability will range between zero (inclusive) and one 
(exclusive), because a maximum instability of one would 
only occur during the theoretical extreme scenario of a non- 
discriminating selection pressure. 

4. SIMULATION AND RESULTS 

A simulated population of agent aggregations, [Ai,Ai,A2, 
...], was evolved to solve user requests, seeded with agents 
from the agent-pool of the habitats in which they were 
instantiated. A dynamic population size was used to ensure 
exploration of the available combinatorial search space, 
which increased with the average size of the population's 
agent aggregations. The optimal combination of agents 
(agent aggregation) was evolved to the user request R, 
by an artificial selection pressure created by a fitness 
function generated from the user request R. An individual 
(agent) of the population consisted of a set of attributes, 
ai,a2, and a user request consisted of a set of required 
attributes, ri,r2, .... So, the fitness function for evaluating 
an individual agent aggregation A, relative to a user request 
R, was 

fitness{A,R) = ^ , ^ , r, (14) 

where a is the member of A such that the difference to the 
required attribute r was minimised. Equation 14 was used 
to assign fitness values between 0.0 and 1.0 to each indi- 
vidual of the current generation of the population, directly 
affecting their ability to replicate into the next generation. 
The evolutionary computing process was encoded with a 
low mutation rate, a fixed selection pressure and a non- 
trapping fitness function (i.e. did not get trapped at local 
optima). The type of selection used fitness-proportional 
and non- elitist, fitness-proportional means that the fitter the 
individual the higher its probability of surviving to the next 
generation. Non- elitist means that the best individual from 
one generation was not guaranteed to survive to the next 
generation; it had a high probability of surviving into the 
next generation, but it was not guaranteed as it might have 
been mutated. Crossover (recombination) was then applied 
to a randomly chosen 10% of the surviving population. 
Mutations were then applied to a randomly chosen 10% of 
the surviving population; point mutations were randomly 
located, consisting of insertions (an agent was inserted 
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Figure 4: Graph of the Probabihties of the Macro- 
States: Mmax and Mhaif at each Generation: The 
system a typical evolving agent population, 
was in the maximum macro-state Mmax only after 
generation 178 and always after generation 482. 
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Figure 5: Visualisation of an Evolving Agent 
Population at the 1000th Generation: The popu- 
lation consists of multiple agent aggregations, with 
each line representing an agent aggregation, and 
therefore each colour representing an agent. 



into an agent aggregation), replacements (an agent was 
replaced in an agent aggregation), and deletions (an agent 
was deleted from an agent aggregation) . The issue of bloat 
was controlled by augmenting the fitness function with 
a parsimony pressure which biased the search to smaller 
agent aggregations, evaluating larger than average agent 
aggregations with a reduced fitness, and thereby providing 
a dynamic control limit which adapted to the average 
size of the individuals of the ever-changing evolving agent 
populations. 

Our evolving agent population (a MAS with evolutionary 
dynamics) is stable if the distribution of the limit proba- 
bilities exists and is non-uniform, as defined by equations 
(10) and (11). The simplest case is a typical evolving agent 
population with one global optimal solution, which is stable 
if there are at least two macro-states with different limit 
occupation probabilities. We shall consider the maximum 
macro-state Mmax and the sub-optimal macro-state Mhaif- 
Where the states of the macro-state Mmax each possess at 
least one individual with global maximum fitness. 



= 1, 



while the states of the macro-state Mhaif each possess at 
least one individual with a fitness equal to half of the global 
maximum fitness, 



thereby fulfilling the requirements of equations (10) and 
(11). The sub-optimal macro-state Mhaif, having a lower 
fitness, is predicted to be seen earlier in the evolutionary 
process before disappearing as higher fitness macro-states 
are reached. The system S will take longer to reach the 
maximum macro-state Mmax, but once it does will likely 
remain, leaving only briefly depending on the strength of 
the mutation rate, as the selection pressure is non- elitist. 

A value of t = 1000 was chosen to represent t = 
oo experimentally, because the simulation has often been 
observed to reach the maximum macro-state Mmax within 
500 generations. Therefore, the probability of the system S 
being in the maximum macro-state Mmax at the thousandth 
generation is expected to be one, PM^ax ~ Furthermore, 
the probability of the system being in the sub-optimal 
macro-state Mhai / at the thousandth generation is expected 
to be zero, p]S,Zu = 0- 



Figure 4 shows, for a typical evolving agent population, 
a graph of the probability as defined by equation (9) 
of the maximum macro- state Mmax and the sub- optimal 
macro-state Mhaif at each generation, averaged from ten 
thousand simulation runs for statistical significance. The 
behaviour of the simulated system S was as expected, being 
in the maximum macro-state Mmax only after generation 
178 and always after generation 482. It was also observed 
being in the sub-optimal macro-state Mhaif only between 
generations 37 and 113, with a maximum probability of 
0.053 (3 d.p.) at generation 61, and was such because the 
evolutionary path (state transitions) could avoid visiting the 
macro-state. As expected the probability of being in the 
maximum macro-state Mmax at the thousandth generation 
was one, PM^ax ~ probability of being in any 

other macro-state, including the sub-optimal macro-state 
Mhaif, at the thousandth generation was zero, PMhaif ~ ^• 

A visualisation for the state of a typical evolving agent 
population at the thousandth generation is shown in Figure 
5, with each line representing an agent aggregation and 
each colour representing an agent, with the identical agent 
aggregations grouped for clarity. It shows that the evolving 
agent population reached the maximum macro-state Mmax 
and remained there, but as expected never reached the 
maximal state of the maximum macro-state, where all the 
agent aggregations are identical and have maximum fitness, 
which is indicated by the lack of total uniformity in Figure 5. 
This was expected, because of the mutation (noise) within 
the evolutionary process, which is necessary to create the 
opportunity to find fitter (better) sequences and potentially 
avoid getting trapped at any local optima that may be 
present. 

4.1 Degree of Instability 

Given that our simulated evolving agent population is 
stable as defined by equations (10) and (11), we can 
determine the degree of instability as defined by equation 
(13). So, calculated from its limit probabilities, the 
of instability was 



= -E 



1000, / lOOOx 



X 

-llogN(X) 



where t = 1000 is an effective estimate for t = oo, as 
explained earlier. The result was as expected because the 
maximum macro-state Mmax at the thousandth generation 
was one, PM^ax ~ probability of being in the 

other macro-states at the thousandth generation was zero. 
The system therefore shows no instability, as there is no 
entropy in the occupied macro-states at infinite time. 

4.2 Stability Analysis 

We then performed a stability analysis (similar to a 
sensitivity analysis) of a typical evolving agent population, 
varying key parameters within the simulation. We varied 
the mutation and crossover rates from 0% to 100% in 
10% increments, calculating the degree of instability^ dins 
from (13), at the thousandth generation. These degree of 
instability values were averaged over ten thousand simula- 
tion runs, and graphed against the mutation and crossover 
rates in Figure 6. It shows that the crossover rate had 
little effect on the stability of our simulated evolving agent 
population, whereas the mutation rate did significantly 
affect the stability. With the mutation rate under or equal 
to 60%, the evolving agent population showed no instability, 
with dins values equal to zero as the system S was always 
in the same macro- state M at infinite time, independent of 
the crossover rate. With the mutation rate above 60% the 
instability increased significantly, with the system being in 
one of several different macro-states at infinite time; with a 
mutation rate of 70% the system was still very stable, having 
low dins values ranging between 0.08 and 0.16, but once the 
mutation rate was 80% or greater the system became quite 
unstable, shown by high dins values nearing 0.5. 

As one would have expected, an extremely high mutation 
rate has a destabilising effect on the stability of an evolving 
agent population. The crossover rate had only a minimal 
effect, because variation from crossover was limited when 
the population had matured^ consisting of agent aggrega- 
tions identical or very similar to one another. It should 
also be noted that the stability of the system is different 
to its performance, because although showing no instability 
with mutation rates below 60% (inclusive), it only reached 
the maximum macro-state Mmax with a mutation rate of 




Figure 6: Graph of Stability with Different 
Mutation and Crossover Rates: With the mutation 
rate under or equal to 60%, the evolving agent 
population showed no instability. 



10% or above, while at 0% it was stable at a sub-optimal 
macro- state. 

5. CONCLUSIONS 

Our extension of the Chli-DeWilde definition of stability 
was developed to provide a greater understanding of MASs 
with evolutionary dynamics^ specifically evolving agent 
populations, including our Digital Ecosystem. We then 
built upon this to construct an entropy-based definition for 
the degree of instability (entropy of the limit probabilities), 
which was also used to perform a stability analysis of a 
simulated evolving agent population. Furthermore, our 
degree of instability provides a definition for the level of 
stability^ applicable to MASs with or without evolutionary 
dynamics. 

Collectively, the experimental results confirm that Chli- 
Dewilde stability has been successfully extended to evolving 
agent populations, while our definition for the degree of 
instability provides a macroscopic value to characterise the 
level of stability. These findings also support the propo- 
sition that Chil-DeWilde stability can be widely applied 
to different types (classes) of MASs, including our Digital 
Ecosystem, which is unique, and for which our simulations 
have shown some of its properties. 

Our extended Chli-DeWilde stability is a useful tool for 
analysing MASs, with or without evolutionary dynamics, 
providing an effective understanding and quantification to 
help better understand such systems. Overall an insight has 
been achieved into the stability of MASs with evolutionary 
dynamics^ including our Digital Ecosystem, which is a first 
step in being able to control such systems. 
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